{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Exploring time varying dynamics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Recall that with differential dynamic microscopy, we find the difference between images separated by some lag time $\\Delta t$:\n", "$$\\Delta I = I(x,y;t) - I(x,y;t + \\Delta t)$$ For a given $\\Delta t$ all such image differences are calculated. For typical DDM analysis, we then Fourier transform each $\\Delta I$ and **average all of the same $\\Delta t$**.\n", "\n", "However, if the dynamics are varying over time, then we might **not** want to average over all times. We instead might want to have a DDM matrix which is a function of lagtime and time (and wavevector too)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Importing the necessary modules" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Modules for plotting\n", "We use [matplotlib](https://matplotlib.org/) for creating figures and plots. Note that we use:\n", "```python\n", "%matplotlib inline\n", "```\n", "This sets the backend of matplotlib to `inline` which means the plots are included in the notebook. If you want the plots to also be interactive (e.g., having the ability to zoom, scroll, and save) then use:\n", "```python\n", "%matplotlib notebook\n", "```" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "import matplotlib\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Modules for numerical work\n", "Here, we import [numpy](https://numpy.org/) and [xarray](https://xarray.pydata.org/en/stable/). \n", "\n", "Note that `xarray` might not have come with your Anaconda Python distribution (or whichever other distribution you installed). If that is the case, you'll need to [install](https://xarray.pydata.org/en/stable/getting-started-guide/installing.html) this package. " ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import numpy as np #numerical python used for working with arrays, mathematical operations\n", "import xarray as xr #package for labeling and adding metadata to multi-dimensional arrays" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Import the PyDDM package\n", "Make sure you append to `sys.path` the directory containing the [PyDDM](https://rmcgorty.github.io/PyDDM/) code. " ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import sys\n", "sys.path.append(\"../../PyDDM\") #must point to the PyDDM folder\n", "import ddm_analysis_and_fitting as ddm" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "## Initializing DDM_Analysis class and computing the DDM matrix\n", "The instance of the `DDM_Analysis` class we create will need, when initialized, metadata about the images to analyze and the analysis and fitting parameters. This can be done by using a [yaml](https://yaml.org/) file as shown in the following cell of code (there, the metadata is saved in the file \"*example_data_silica_beads.yml*\". " ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Provided metadata: {'pixel_size': 0.242, 'frame_rate': 41.7}\n", "Image shape: 3000-by-128-by-128\n", "Number of frames to use for analysis: 999\n", "Maximum lag time (in frames): 600\n", "Number of lag times to compute DDM matrix: 40\n", "Applying binning...\n", "Dimensions after binning (999, 64, 64), the new pixel size 0.484\n" ] } ], "source": [ "#The yaml file `example_data_silica_beads.yml` contains the metadata and parameters above\n", "#Note that here we *are* applying a 2x2 binning\n", "\n", "ddm_calc = ddm.DDM_Analysis(\"../../Examples/example_data_silica_beads.yml\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Below, with the method `calculate_DDM_matrix`, we compute the DDM matrix and some associated data. The data will be stored as an [xarray Dataset](https://xarray.pydata.org/en/stable/generated/xarray.Dataset.html) as an attribute to `ddm_calc` called `ddm_dataset`.\n", "\n", "**Note**: There are a few optional arguments we can pass to `calculate_DDM_matrix`. There is an optional argument `quiet` (*True* or *False*, *False* by default). Then we have some optional keyword arguments (all of which could also be specified in the YAML file). These are: `overlap_method` which sets the degree of overlap between image pairs when finding all image differences for a each lag time and is either *0*, *1*, *2*, or *3*, `background_method` which sets how to estimate the background parameter *B* and is either *0*, *1*, *2*, or *3*, and `number_lag_times`. If any of these three keyword arguments are set here, the value specified in the YAML file will be overwritten. " ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The file C:/Users/rmcgorty/Documents/GitHub/PyDDM/Examples/images_nobin_40x_128x128_8bit_ddmmatrix.nc already exists. So perhaps the DDM matrix was calculated already?\n", "Do you still want to calculate the DDM matrix? (y/n): n\n" ] } ], "source": [ "#If a file is found that matches what, by default, the program will save the computed DDM matrix as, \n", "#then you will be prompted as to whether or not you want to proceed with this calculation. If you \n", "#do not, then enter 'n' and the program will read from the existing file to load the DDM dataset \n", "#into memory. If you enter 'y', then the DDM matrix will be calculated.\n", "\n", "ddm_calc.calculate_DDM_matrix()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Finding variability in the DDM matrix" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When computing the DDM matrix, we average over all pairs of images separated by the given set of lag times. For example, for a lag time of 2 frames, we will find the differences between frames 1 and 3, 2 and 4, 3 and 5, etc. The power spectrum of those differences are then taken to yield the DDM matrix. But what if the dynamics in the sample are changing? For example, perhaps the difference between frames 1 and 3 will be very different from the difference between frames 1001 and 1003. " ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "tlags = np.arange(1,998,dtype=int) #We will compute the DDM matrix for all lag times from 1 frame to 998 frames\n", "\n", "#With the function 'variationInDDMMatrix', we compute the DDM matrix for the given lag times\n", "# **without** doing any averaging over time. With the optional paramter 'save_full_ddmmat' set to \n", "# False, we will be radially averaging the 2D DDM matrix. This will save memory. \n", "ddm_variability = ddm_calc.variationInDDMMatrix(tlags, save_full_ddmmat=False)" ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [], "source": [ "ddm_variability.to_netcdf(\"ddm_variability_silicabeads.nc\") #save generated dataset to disk" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
<xarray.Dataset>\n", "Dimensions: (lagtime: 997, time: 998, q: 32)\n", "Coordinates:\n", " * time (time) float64 0.0 0.02398 0.04796 ... 23.86 23.88 23.91\n", " * lagtime (lagtime) int32 1 2 3 4 5 6 7 ... 992 993 994 995 996 997\n", " * q (q) float64 0.0 0.2028 0.4057 0.6085 ... 5.882 6.085 6.288\n", "Data variables:\n", " ddm_matrix (lagtime, time, q) float64 0.0 220.0 184.4 ... nan nan nan\n", " alignment_factor (lagtime, time, q) float64 nan 6.123e-17 ... nan nan\n", "Attributes:\n", " AlignmentFactorAxis: 0
array([ 0. , 0.023981, 0.047962, ..., 23.860911, 23.884892, 23.908873])
array([ 1, 2, 3, ..., 995, 996, 997])
array([0. , 0.20284 , 0.405681, 0.608521, 0.811362, 1.014202, 1.217043,\n", " 1.419883, 1.622723, 1.825564, 2.028404, 2.231245, 2.434085, 2.636926,\n", " 2.839766, 3.042607, 3.245447, 3.448287, 3.651128, 3.853968, 4.056809,\n", " 4.259649, 4.46249 , 4.66533 , 4.86817 , 5.071011, 5.273851, 5.476692,\n", " 5.679532, 5.882373, 6.085213, 6.288053])
array([[[ 0. , 220.04545857, 184.38399034, ...,\n", " 94.58914071, 93.7690884 , 81.66854128],\n", " [ 0. , 112.3428197 , 60.96815397, ...,\n", " 66.38218294, 53.6350686 , 62.51979943],\n", " [ 0. , 98.41897164, 101.97738069, ...,\n", " 54.49735033, 55.10627058, 52.23193462],\n", " ...,\n", " [ 0. , 77.70965475, 135.90005735, ...,\n", " 58.92433517, 58.4778158 , 55.18350923],\n", " [ 0. , 89.67129696, 123.87543033, ...,\n", " 60.85931764, 52.93569647, 49.83894966],\n", " [ 0. , 147.59986325, 153.57530257, ...,\n", " 53.08847067, 57.61446569, 55.32019147]],\n", "\n", " [[ 0. , 373.29665834, 216.51303491, ...,\n", " 81.99647868, 109.95927323, 101.96291398],\n", " [ 0. , 116.57465737, 105.10468156, ...,\n", " 76.24483146, 63.19264613, 68.05139254],\n", " [ 0. , 247.78469075, 147.30543405, ...,\n", " 61.80097638, 65.04270436, 49.63940288],\n", "...\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan],\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan],\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan]],\n", "\n", " [[ 0. , 1512.98915149, 3566.39329903, ...,\n", " 119.86221091, 110.30273635, 113.88357923],\n", " [ 0. , 1391.98865504, 2736.34523223, ...,\n", " 129.87626829, 110.08957292, 100.17179273],\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan],\n", " ...,\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan],\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan],\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan]]])
array([[[ nan, 6.12323400e-17, -9.80384669e-02, ...,\n", " 2.85253420e-02, -9.50111080e-02, -1.57947399e-01],\n", " [ nan, 6.12323400e-17, -2.49717851e-01, ...,\n", " -5.26666998e-03, 8.37042726e-02, -5.91464131e-02],\n", " [ nan, 6.12323400e-17, 1.45570391e-01, ...,\n", " 2.90914553e-02, 5.51512863e-02, -2.96649245e-02],\n", " ...,\n", " [ nan, 6.12323400e-17, -1.93642012e-01, ...,\n", " -6.88032939e-02, -3.55367564e-02, 1.01651029e-01],\n", " [ nan, 6.12323400e-17, 1.28455034e-01, ...,\n", " -5.07279273e-02, 1.22883479e-01, -1.21307943e-01],\n", " [ nan, 6.12323400e-17, -3.23709260e-01, ...,\n", " -8.71467418e-02, -7.72352729e-02, -1.05252909e-01]],\n", "\n", " [[ nan, 6.12323400e-17, -1.25068532e-01, ...,\n", " -8.48296229e-02, -2.46594244e-03, 8.77949563e-03],\n", " [ nan, 6.12323400e-17, 2.06295396e-01, ...,\n", " -1.55475424e-01, -1.33299485e-01, -9.02193195e-03],\n", " [ nan, 6.12323400e-17, 1.97397855e-01, ...,\n", " 1.47180620e-01, 1.13735351e-01, -1.06306659e-02],\n", "...\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan],\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan],\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan]],\n", "\n", " [[ nan, 6.12323400e-17, 2.90721919e-01, ...,\n", " 4.76096954e-02, -3.37923445e-03, 1.07607578e-01],\n", " [ nan, 6.12323400e-17, 1.65293491e-01, ...,\n", " -1.30611866e-01, -8.31035597e-02, 1.87609867e-03],\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan],\n", " ...,\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan],\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan],\n", " [ nan, nan, nan, ...,\n", " nan, nan, nan]]])